DEVELOPMENT AND DESIGN SPACE EXPLORATION OF DEEP CONVOLUTION NEURAL NETWORK FOR IMAGE RECOGNITION

CHAPTER ONE

Introduction

ntroduction to Neuro-Inspired Systems

Neuro-Inspired systems are computing systems which are modelled after biological neuro- processes in the brain of animals, especially higher animals like humans. Such processes have cognitive abilities which make the animal intelligent, in the sense that they are able to make decisions having processed data they receive from their environment or with changes involved in their body system. Biological beings are not born with the ability to make decisions after perceiving stimuli but they attentively perform an activity when they continually process such novel stimulus; and such neurological cognition is the ability that neuro-inspired systems want to leverage on and mimic, if not surpass.

Artificial intelligence (AI) is a branch of computer science that emphasizes the creation of intelligent agents that work and perceive as humans do (Techopedia, 2017). Machine learning (ML) algorithms are learning algorithms for AI and have to do with the mathematical models used to build software that progressively modifies its algorithms so as to improve future results, that is software with the ability to learn. These algorithms are different from the conventional sequential algorithms mostly used in computing in the fact that they are not deterministic algorithms. Examples of machine learning algorithms are supervised learning, unsupervised learning, reinforcement learning, transduction, learning to learn, semi-supervised learning algorithms.

Artificial neuro-science deals with artificial neural networks (ANNs) which are an ML method with the goal of building cognitive algorithms which are closely modelled after neural networks in the human brain (a connectionist network). Deep learning has to do with perceiving many levels of abstraction and representation that help a person to make sense of datasets such as images, sound, and text (Ng et al., 2015). It is composed of more complex algorithms because it is an advanced form of an ANN.

Computer vision is the science of endowing computing machines with visual perception, that is, the ability to see (Henrique & Pinheiro, 2017). A convolutional neural network (CNN) is a variant of ANN that possesses a deep learning architecture which is a neural network architecture with a stack of more than two or more non-linear layers (Ian, 2016). The eyes are described as being the light of the body or the window of the soul, the soul being an embodiment of thought or perception. In computer vision, the processing elements emulate the activity of neuronal cells in the visual cortex of the human eye and their deep complex architecture, where some neurons activate when low-level features are seen, while other neurons which have a wider receptive field activate when more abstract or high-level features are being viewed. Computer vision – an image recognition system with the aid of convolutional neural networks – is the focus of this thesis due to this deep neuro-inspired architecture.

​​​​​​​Statement of the Problem

Most computing algorithms are deterministic or sequential in nature. A sequential algorithm is an algorithm that follows a specific set of rules to accomplish a task. But not all computing problems can be solved with deterministic algorithms. Deterministic algorithms do not have cognitive abilities, meaning that they cannot learn from examples provided and use the information gained to identify patterns.

The Von Neumann architecture of computing does not possess true parallelism due to the processing latency it experiences between the memory and the central processing unit, but a neural networking system eliminates the use of such an architecture and it is regarded as possessing true parallelism.

An ANN can be a software or hardware implementation. A software implementation of an ANN suffers the bottlenecks of bandwidth and use of too much memory. Consequently, the hardware implementation is a massively parallel system which overcomes such bottlenecks. A software or hardware implementation of a neural network can still have cognitive abilities and can solve problems that sequential algorithms cannot.

​​​​​​​Motivation

Deep learning became a sensation in its use for classification of objects using deep CNNs (DCNNs) in the year 2012 when Krizhevsky achieved impressive results in the yearly ImageNet image classification contest (Krizhevsky, Sutskever, & Hinton, 2012). The computational power using parallel processing computational units and graphical processing units have further advanced research using deep learning methods. DCNNs are able to learn feature representations for novel datasets, they behave well when they are trained using many datasets and have adhered to state of the art performance compared to other computer vision approaches. Computer vision can be applied in industry and find various applications in our daily lives, like the production of driverless cars, autonomous aerial vehicles and can be used in medicine to identify pathogens and ailments like cancer with very high success (Davy et al., 2013). Figure 1.1 depicts the vision of a CNN system and that of a human eye.

 

Figure 1.1: Object Recognition in Machines and Humans

​​​​​​​Research Objective

ANNs are of various kinds and can be applied in various aspect of life. One of the objectives of this thesis is to gain a deep understanding of neuro-inspired computing systems and their various applications such as in weather predictions, in building recommender systems, in predicting stock exchange markets, and building autonomous systems. The main objective of this thesis to gain a deep understanding of DCNNs that are used for image recognition and build an efficient algorithm that can be used for object recognition and classification and perform deep space explorations by implementing software emulations to determine the best CNN model that can be used for an image classification based the performance of its image classification accuracy. The deep space exploration is achievable because the design of the algorithm for our CNN model provides us with the flexibility to adjust some of its parameters so as to observe the results of its performance in image classification. The optimized classifier system should be used for software emulations that exhibit neuronal processing in the visual cortex for object recognition of static and real-time images just as it is possible in humans, with little room for errors caused by objects in an image exhibiting variations such as pose, view pose, illumination, appearance and occlusion. Visual recognition is a difficult computational problem and it will be significantly involved in making intelligent systems (Pinheiro & Henrique, 2017). A DCNN (Yann, Haffner, Bottou, & Yoshua, 2012) has an architecture that has surpassed others when it comes to the use of image-based applications for computer vision. It is able to reliably identify objects in an image irrespective if its position and pose within the image dataset.

​​​​​​​Significance of Study

Object recognition using DCNN machine learning models is of importance as a study because of the diversity it can cut across in its application. Before a cognitive agent is able to make decisions in a live environment, its first point of call is an observation of its surroundings, which is possible with computer vision. DCNNs can be applied in building autonomous machines, speech recognition and natural language processing. DCNN models are cheap to implement. The only downside is the time taken to train the network models, but once they are implemented, they can be made available on the internet for use by millions of people that require their features.

​​​​​​​Thesis Outline

​​​​​​​Chapter 1 – Introduction to Neuro-inspired Systems

This chapter introduces the definition of neuro-inspired systems, using neuro-inspired systems for image recognition, its objects and goals.

​​​​​​​Chapter 2 – Literature Review (Related work about Image Recognition)

Related work about Image recognition agents is discussed in this chapter.

​​​​​​​Chapter 3 – Study of Image Recognition Algorithm Using Supervised Learning Algorithms

Supervised machine learning algorithms such as backpropagation will be discussed in this chapter and results from software emulations of the CNN models tested will be documented as well.

​​​​​​​Chapter 4 – Evaluation and Discussion

In this chapter, the software model being developed is evaluated against the objectives set in Chapter one of this project, to see if it meets up with the requirements and goals that were initially set.

​​​​​​​Chapter 5 – Conclusion and Future Work

The purpose of this chapter is to see if the project has added more knowledge and functionality to similar previous works being carried out to evaluate the effects of using other activation functions such as Sigmoid, Tanh in the activation layer in a chosen CNN model and also the effect of using different kernel sizes in the convolution layer of a chosen CNN. Future work regarding the software model is also discussed.